Metrics for Predicting Run-Time Failures and Maintenance Effort: Four Case Studies

August 1998 The distribution of faults within any particular software product is not uniform. For example, 47 percent of the faults found by users of OS/370 were associated with only 4 percent of the modules [1]. Therefore, what is needed are metrics that can be applied to detailed designs to predict which modules will be faultprone. Then, instead of wasting money detecting and correcting faults during implementation and integration, those modules can be redesigned before coding commences. Metrics of this kind can also be applied to existing software, to predict which modules are likely to contain residual faults when the software is installed. Residual faults can have two major adverse consequences: run-time failures, which prevent users from making optimal use of the software; and time that has to be spent on corrective maintenance. If a fault is detected in an existing module that has been flagged as fault-prone, it may turn out cheaper to discard, redesign, and recode the module than to attempt to fix one fault in a module that probably contains many other faults. The CDM is a metric that accomplishes this objective. It has outperformed a wide variety of other metrics, both classical and object-oriented, in predicting run-time failures and residual faults [2]. In this article, we compare the performance of CDM with other metrics on four real-world case studies: run-time failure data for a COeach module. We then chose an objective measure of software quality, for example, the number of faults detected in a module after installation. Next, we computed the Spearman rank correlation [2] between the LOC in each module and the number of faults in each module. A high correlation would indicate that modules with many LOC also have many faults, i.e., a high correlation would mean that lines of code is a good predictor of faults.